video motion detection (VMD) is the backbone functionality for most Intelligent Video Surveillance (IVS) systems, and the robustness of the system is one of the primary performance criteria for an IVS system. For without a robust motion detection engine, the subsequent functionalities in an IVS system will not be accurate or reliable. One of the key factors in enhancing the robustness of dynamic video analysis is to provide an accurate and reliable means for shadow detection. For if left undetected, shadow pixels can cause problems such as object merging and object shape distortion, and may result in incorrect object tracking and classification.
Most of the shadow tracking algorithms known in the art are computationally intensive—some to the extent of equaling the computational requirements of the motion detection algorithm. Many of these known shadow detecting algorithms take into account a priori information, such as the geometry of the scene or of the moving objects, and the location of the light source. Algorithms that don't require any such a priori information exploit such things as spatial information (e.g., shadows in an image frame are attached to their respective object), transparency (i.e., a shadow always makes the region it covers darker; this involves the appearance of single pixels), and/or homogeneity (the ratio between pixels when illuminated and the same pixels under shadow is roughly linear).
One method of shadow detection known in the art computes an intensity ratio between the current image and the reference image for each pixel within the detected blobs. This method uses the characteristic that the photometric gain of a shadow with respect to the background image is less than unity and roughly constant over the whole shadow region, except at the edges (the penumbra region). In this method, a priori assumptions regarding certain shadow identification rules result in the detection only of shadows with quite a large area with respect to the object itself.
In another method of shadow detection known in the art, the shadow detection algorithm initially decomposes the difference between the background image and the current image into brightness and chromaticity components. Then, a preset threshold is applied on the separate components. This yields a pixel classification into background, shadow or foreground categories.
In another method known in the art, the method is applied to gray level images taken by a stationary camera. The method uses the property of a moving cast shadow that its illumination change (measured directly from two frames using a physics-based signal model of the appearance of a shadow) is smooth. The system prepares two distinct modules to detect penumbra and shadows separately. The first module uses a two-frame difference between subsequent frames as the input image. A linear luminance edge model is applied to detect likely shadow boundaries. Further, a Sobel operator is measured perpendicularly to the borders and the results are made to be thresholds using both a gradient outcome and an edge model. The second module computes the ratio between two subsequent images and thresholds on the local variance.
Another shadow detection approach known in the art is based on the similarity of background and shadow regions. A color segmentation method using a K means algorithm segments the pixels as a cast shadow, a self-shadow, or a dark object. This is followed by a connected component analysis to merge the similar regions. A gradient comparison between the background image and the shadow pixels gives a validation for removing the shadow.
Another shadow detection approach known in the art explores the HSV (Hue, Saturation, and intensity Values) color space. The hue, saturation & intensity values are checked individually in order to determine if the values lie between certain ranges. A shadow mask is thus formed having the values 1 or 0 based on the conditions satisfied. A difficulty with this method is in fixing the range values for H, S and V.
All of the aforementioned techniques of the prior art suffer from one or more shortcomings. Those techniques that are based on the homogeneity property of shadows assume that the ratio between pixels when illuminated and when subjected to a shadow is constant. In fact however, that ratio is highly dependent on illumination in the scene, and shadow correction therefore is not effective in these prior art systems where the ratio is assumed to be constant. Additionally, approaches that employ techniques like multi-gradient analysis to remove the penumbra region of a shadow that is left by image division analysis are computationally intensive. The art is therefore in need of an improved system and method to process shadow information in image data.